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Abstract: This paper proposed a robust, radial and effective content-based image retrieval (CBIR) 
or query by image content (QBIC) or content based visual information retrieval (CBVIR) approach, 
which is based on colour, texture and shape features. Due to the enormous increase in image 
database sizes, as well as its vast deployment in various applications, the need for CBIR development 
arose. In this proposed approach, image attributes like image name, keywords and meta data are not 
used to compute image similarity and image retrieval. So, concept based image retrieval is not used. 
If an image is given as an input query and the output is based on the input image query, it is called as 
reverse image search. So, images can be searched based on their contents (pixels) but not by their 
keywords. It is difficult to measure image content similarity due to visual changes caused by varying 
viewpoint and environment. In this paper, a simple and efficient method to effectively measure the 
content similarity from image measurements is proposed. The proposed approach is based on the 
three well-known algorithms: colour histogram, texture and moment invariants. It ensures that the 
proposed image retrieval approach produces results which are highly relevant to the content of an 
image query, by taking into account the three distinct features of the image and similarity metrics 
based on Euclidean measure. Colour histogram is used to extract the colour features of an image. 
Gabor filter is used to extract the texture features and the moment invariant is used to extract the 
shape features of an image. It also uses fuzzy similarity measures. 

Keywords: reverse image search, concept based image retrieval, content based image retrieval, 
Euclidean measure, histogram, Gabor filter, moment invariant. 



I. Introduction 

In recent years, the availability of large, unordered image collections on the Internet has given rise to a 
need for robust, efficient, effective and scalable image retrieval systems. With the development of digital image 
processing technology, it has become imperative to find a method for efficient image retrieval. One of the most 
interesting (comparatively) new methods currently being developed around the web revolves around the idea of 
using images as a basis for search queries. Using an image as an input query and getting effective results by 
analyzing its contents is still under research in image processing. Image processing is a method to convert an 
image into digital form and perform some operations on it, in order to get an enhanced image or to extract some 
useful information from it. Image processing is also a form of signal processing for which the input is an image, 
such as a photograph or video frame; the output of image processing may be either an image or a set of 
characteristics or parameters related to the image. 

In the traditional methods, searching, comparing and retrieving of images are based on the text 
attributes such as keywords, metadata related to that image. In our proposed system, the visual contents of an 
image such as colour, texture and shape are analysed to produce effective results. Content based image retrieval 
method is used to analyse the image contents. Content based methods are still under development. 

Different researchers proposed various algorithms in order to address the problem of image retrieval 
using content based approaches. Mostly, those algorithms rely on one algorithm and ignore the existence of 
others. Approaches based on one specific algorithm (ie) colour, texture and shape can work effectively only on 
specific types of images. When different types of images are input to these systems their performance is 
degraded. For example, approaches based on colour histogram take into account only the visual contents relating 
to colours and ignore shape and texture. Similarly, approaches based on shape perform reasonably well when 
dealing with the shape of images without taking into account colour histogram and texture but do not produce 
effective output for all set of images. 
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The proposed system will take into account all the visual features of an image like color, texture and 
shape. So, the output produced by the system is efficient and effective. Typically, a large-scale retrieval system 
focuses on the following: (i) image representation, (ii) image similarity measure and (iii) refinement of retrieval 
results. 

Analyzing the contents of an image is still under development. In Google images, mostly people 
specify text or keywords to retrieve images from the database. In June 2011, searching and comparing by 
images contents is permitted. The user can search by an image in Google images. Google images will analyse 
the contents of an image query and produce the desired results which contains the information about the image 
and also specify in which websites that image is available. But in Yahoo images, Bing images and Ask images, 
searching and comparing by images contents is not permitted and the user can specify only the text attributes to 
retrieve the images. This shows that using the image as an input query is a new emerging method. The following 
table shows different image retrieval systems and their characteristics. 



IMAGE 
RETRIEVAL - 


INPUT 


OUTPUT 


SYSTEM 


Text 


Image 


Google 
Images 


Allowed 


Allowed 


A Set of Related Images 


Yahoo 
Images 


Allowed 


Not Allowed 


A Set of Related Images 


Bing 
Images 


Allowed 


Not Allowed 


A Set of Related Images 


Ask 
Images 


Allowed 


Not Allowed 


A Set of Related Images 



II. Existing System 

In the existing system, the input can be either text or metadata or colour. If the user specifies an input, 
then the input is compared with the image database. Based on the input, the results are retrieved from the 
database. The output will contain the image which has attributes similar to the input. Images are not compared 
by its contents (pixels). The images are compared by its attributes like text. So, the existing system uses Concept 
based or Description based or Text based image indexing/retrieval. The existing system will not consider the 
major characteristics of an image like colour, texture and shape. Only the colour is considered. 
The existing system can be defined as follows: 

> Compare by text 

> Compare by colour 

> Compare by text and colour 

1. Compare by Text: 

It is predominantly and traditionally used method to search, compare and retrieve images from the 
database. In this method, images are searched and compared based on the attributes of the image. The attributes 
of the image include image name, identifier (ID), keywords related to that image, metadata, title, subject. This 
method retrieves all the images from the database whose attributes are similar to the input query and those 
images are displayed to the user. This method works well with any set of image database. 

2. Compare by Colour: 

In this method, the user can specify only the colour as an input. The results will be produced based on 
the colour given as an input. If the images contain the specified colour, then it is included in the output. The user 
cannot specify the image attributes like keywords, metadata as an input. 

3. Compare by Text and Colour: 

In this method, both text and colour are given as an input query. The output will be produced based on 
the user's input. There are two approaches. 

a) Compare by Text and Single Colour: 

In this method, text and colour are given as an input. Multiple colours cannot be specified. Only single 
color can be specified. 
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b) Compare by Text and Multiple Colours: 

In this method, text and multiple colours are specified as an input. The results will be based on the text 
and colours specified by the user. 

2.1. Problems Related With Existing System 

y If we don't know any details or description about an image, then we cannot use the existing system since it 
uses Description based image indexing or retrieval. 

> Reverse image Search is not used so image cannot be given as an input query. Only the attributes of an 
image like text or keywords can be given as an input. 

> Major characteristics of an image like colour, texture and shape are not considered while producing the 
output. 

> The pose and expressions of a person in an image are not considered in the existing system. 

> Similar images cannot be produced. 

III. Proposed System 

In the proposed system, Reverse by Image Search method is used. In this method, images are searched 
by its contents (pixels). Since the contents of an image are used for comparison, it can be called as Content 
based image retrieval (CBIR) or Content based Visual Information Retrieval (CBVIR). 

Since the image is given as an input query, CBIR can also be called as Query by Image Content 
(QBIC). CBIR does not use the attributes of an image like keywords or metadata for comparison and producing 
the desired results. CBIR system mainly focuses on the major characteristics of an image like colour, texture and 
shape. A desirable output is produced from the large collection of images stored in the database based on their 
visual contents. 

Image is captured and it is given as an input. If the image is taken in an improper light source which is 
because of environmental factors like fog or snow or at midnight, the image may not be clear and quality may be 
reduced and it may obscure the details of an image. In this case, image preprocessing can be done. The 
brightness of an image can be adjusted. To increase or reduce the brightness of an image, all the values of pixels 
must be changed. To increase the brightness, add some constant value to each and every pixel. To decrease the 
brightness. Subtract some constant value from each and every pixel. This will enhance the contents of an image 
and improve its quality and resolution can be increased. 

This enhanced input image is used by the system to extract visual features from the image. The visual 
features include shape, colour and texture. These features are examined in order to search, compare and retrieve 
similar images from the image database. 

The similarity of visual features between query image and each image in a database is calculated based 
on their distance by comparing the feature vectors of two images. To calculate the similarity, an image distance 
measure is used which uses the visual feature of the images, If the image distance measure is 0, an exact match 
of input image is found in the image database. If the value is greater than 0, an exact match is not found and 
only similar images are retrieved and displayed. The image retrieval system displays images, as the result of an 
image query that have the closest similarity and also displays similar images and the details of the image will 
also be displayed. 

If the input image matches exactly with the image stored in the database then that image will be 
retrieved and displayed to the user. Even though if the input image does not match exactly with the input image, 
similar images will be displayed. 
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Fig 1: Overall Representation of the Proposed Approach 
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3.1. BUILDING BLOCKS OF THE PROPOSED APPROACH 

3.1.1 Colour Features 

Colour feature is the most commonly used visual feature for image retrieval. Many colour models are 
available that can be used to represent images such as HSI, HSV and YCbCr. Colours play a major role in 
human perception. The most commonly used colour model is RGB, where each component represents red, green 
and blue and these are primary colours. Several millions of colours can be formed by combining primary 
colours. RGB colour space is based on RGB colour model. This colour space defines how colours are 
represented as tuples (ordered list) of numbers. Images displayed on monitor are defined in RGB colour space. 
There are different ways to use 

colour for image retrieval purpose such as colour histogram, colour moment and colour coherence. But the most 
effective method is colour histogram. 

Colour histogram is a representation of the distribution of colours in an image. It describes the 
proportion of colours in each pixel. The colour histogram provides meaningful information for measuring the 
similarity between two images, as it is robust against object distortion and scaling of the object. Additionally 
high effectiveness, computational simplicity and low storage requirements possibility makes it best among 
others. Due to these characteristics, many researchers have started to use histogram based colour image retrieval 
method. 

Colour histogram is used to extract colour from any kinds of image formats like jpeg, bmp, png. Every 
pixel in an image is basically represented as a point in RGB colour model. This colour point is represented by 
three values that hold the colour information. Suppose if the extracted colour is red, the RGB values can be 
255,0,0. If the colour is violet, the values are 255,0,255 and for white colour, values are 255,255,255. 

3.1.2 Texture Features 

Texture is another important feature of an image that can be extracted for the purpose of image 
retrieval. Image texture refers to surface patterns or visual patterns which show granular details of an image. It 
also gives information about the spatial arrangement of different colours or intensities in an image. 
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There exist two main approaches for texture analysis. They include structural and statistical 
approaches. In structural texture approach, the surface pattern is repeated such as floor design that contains the 
same pattern. In statistical texture, the surface pattern is not regularly repeated in the same pattern such as 
different flower objects in a picture that normally contains similar properties but not exactly the same 

Texture information can be extracted from an image using co-occurrence matrix. Co-occurrence matrix 
is a popular representation of texture feature of an image. It is constructed based on the orientation and distance 
between image pixels. Co-occurrence matrix captures the numerical features of a texture using the spatial 
relations. Numerical features computed from the matrix can be used to represent, compare and classify the 
textures. 

Gabor filters were found to be very effective in texture analysis. Gabor filter is used in various type of 
applications due to its effectiveness in the area of texture based image analysis. 

3.1.3 Shape Features 

Shape feature plays a vital role in object detection and recognition. Object shape features provide 
robust and efficient information of objects in order to identify and recognize them. Shape features are 
considered very important in describing and differentiating the objects in an image. 

Shape features can be extracted from an image by using two kinds of methods: contour and regions. 
Contour based methods are normally used to extract the boundary features of an object shape. Such methods 
completely ignore the important features inside the boundaries. 

Region-based methods that rely on shape descriptors are normally able to extract both kinds of 
features: boundary and region. These provide valuable information to represent the shape of an image for feature 
extraction. Region-based image retrieval methods firstly apply segmentation to divide an image into different 
regions/segments, by setting threshold values according to the desirable results. On the other hand the boundary 
of an image can be obtained by applying any edge detection method to an image. 



Retrieval Approach 

Fig 2: PROPOSED CONTENT BASED IMAGE RETRIEVAL APPROACH 
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3.2 STEPS INVOLVED IN THE PROPOSED APPROACH 

Step 1: An image histogram can be generated as follows: 

A digital image is commonly seen as a 2D mapping / : x— * v from M x N pixels x = [i, j]T to values v (where i 
=1, 2, . . . ,M and j = 1, 2, . . . , N correspond to y-axis and x-axis respectively). Often the values v are discrete 
intensity values in the range [0-255]. The image histogram is calculated as: 

h b =2Xi2f=i^a;),V 6 =0, 1,2... (1) 

where S b (i, j) = 1 if the v at pixel location [i, j] falls in b, and S b (i, j) = 0 otherwise. 
Step 2: To extract the texture features: 

Step 2(a): A 2D Gabor function consists of a sinusoidal plane wave of some orientation and frequency, 
modulated by a 2D Gaussian. The Gabor filter in spatial domain is given below: 

g,X,eXo,y (x,y) = exp{- %2+ 2 ^ 2 y ) cos(2tt j + V) 

" (2) 

where x = x cos(0) + y sin(0) 
y = y cos(0) - x sin(0) 

In the above equation, wavelength of cosine factor is represented by A; 0 represents the orientation of the normal 
to parallel stripes of a Gabor function in the degree; the phase offset in degree is represented by if; the spatial 
aspect ratio which specifies the elliptically of the support of the Gabor function is represented by y ; and a is the 
standard deviation of the Gaussian that determines the linear size of the receptive field. 

Step 2(b): When an image is processed by Gabor filter; the output is the convolution of the image I(x, y) with 
the Gabor function g(x, y) which is 
r(x, y) = I(x, y) *g(x, y) (3) 
where * represents the 2D convolution. 

Step 2(c): After applying Gabor filters on the image by orientation and scale, we are able to obtain an array of 
magnitudes 



E(m, n) =£j.£ y 



G mn O,y) 



m = 0, 1,...,M- l;n = 0, 1...JV-1 (4) 

Step 2(d): The following mean u nin and standard deviation a mn of the magnitude of the transformed coefficients 
are used to represent the texture feature of the region: 
u irm = E(m, n)IP xQ (5) 

a m „=^I. x I. y (\G mn (x,y)| - H MN Y/P*Q (6) 

where M represents the scale and ,/V represents the orientation. 

Step 3: Calculate the descriptions of shape features which are independent of location, orientation and size. 
Step 3(a): The 2D moment of order (p +q) of a digital image /(x, y) is defined as 
m m = -L x X y xPy« f(x,y) (7) 

for p, q = 0, 1,2 where the summations are over the values of the spatial coordinates x and y spanning the image. 

Step 3(b): The corresponding central moment is defined as 

p pq = S x S y (x - W (y - y) q f(x,y) (8) 

x = mio/m 00 , y = m 0 i/m 0 o 

where x, y are called the centre of the region. 
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Step 3(c): The normalized central moment of order (p+q) is defined as 

Tlp,q = Hp,q/^0,0 ( 9 ) 

for p,q = 0, 1, 2,... where J=[(p +q)/2]+l. 

Step 4: To calculate the similarity, we use Euclidean distance between two feature vectors as follows. 
ED(M k ,M')= Jl]r=i( M f - M f) 2 (10) 

where M 1 and M' are image query and image database respectively, i is a feature range. Closer distance 
represents the higher similarity between images. 

3.2.1 Fuzzy Similarity Measure 

We use fuzzy heuristics to measure similarity between the query image and the database images in 
order to retrieve and display relevant or similar results to the user query. There are three types of preferences 
that are taken into account while checking the similarity between images. The first priority is given to the shape 
features, as shape of an image is not easily affected by external factors, and also it is invariant to the rotation, 
translation and orientation The second priority is given to the colour features, as these features are invariant to 
the rotation and translation. The third priority is given to the texture features. By defining these criteria along 
with the fuzzy rules, we assume that better results can be achieved using our proposed approach. The Mamdani 
fuzzy inference method is used to perform fuzzy rules in our proposed approach. 

X, Y and Z are three visual features of images that have been retrieved using Moment Invariants, 
Histogram of images(Red, Green, Blue and Intensity) and Gabor wavelet. Based on the Euclidean distance 
algorithm we find the images that have high similarity between query and database image. After obtaining the 
relevant images to the query image, we need to find common images between X, Y and Z set of images. The 
common set of images are considered the most relevant images. Commonality is measured using the below 
criteria. 

X = Shape features are used to calculate the distance between query and database image 
Y= Colour features are used to calculate the distance between query and database image 
Z= Texture features are used to calculate the distance between query and database image 
S= Image similarity 

Several fuzzy rules can be used: If shape is low and colour is low and texture is low then similarity is 
high. If shape is low and colour is high and texture is low then similarity is medium. If shape is high and colour 
is low and texture is high then similarity is low. If shape is high and colour is high and texture is low then 
similarity is low. 

3.3 ADVANTAGES 

The advantages of the proposed system are discussed below: 

^ The proposed system can be used if we don't know any details or description about an image since it uses 
Content based image retrieval. Only the contents of an image are analyzed to produce effective results and 
the text attributes like metadata, keywords are not used. 

> An image can be rotated to several angles and it can be given as an input. But still the image database 
contains the images which are not rotated. The rotated input image is compared with the database and 
results are produced. Suppose image il is stored and it is rotated to 180 degree and given as an input query. 
Even though input il is rotated and it differs from the database image, both are similar according to our 
system. 

^ Cropped versions of images also work in our proposed system. Suppose an image i2 contains three persons 
and stored in the database. Now image i2 is cropped and it contains only one person and given as an input. 
Even though both images differ, our proposed system will identify the uncropped image stored in the 
database and gives the effective results. 

> Even though if the background colour of an input image differs from the images stored in the database, our 
system will produce effective results. Suppose the background colour of an image i2 stored in the database 
is blue. In the input query, it's background colour is changed to some other colour say white, then also our 
system will produce better results. 
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> The pose and expressions of a person in an image may vary from time to time but this will not affect the 
effective output produced by the system. The pose and the expression in the input image may vary from the 
pose and the expression of the images stored in the database. 

> Location, orientation and size of the image may not affect the output produced by the system. 

> This system works well on black and white, colour and grayscale images. 

> Image can be given as an input query. The output will be produced based on the input query since reverse 
image search is used. 

> Images which are similar to input can be produced as an output. If there is an exact match that image will be 
displayed or similar images will be displayed. 

> This system accepts all kinds of image formats (jp e g> P n gi bmp) and dimensions (103*151, 184*274 etc). It 
works well for some gif images. 

> This system is fully automatic and does not require any human intervention to extract the visual features of 
an image and to produce an effective output. 

> To find information about unidentified products and other objects. 

3.4 APPLICATIONS 

The Proposed system has several applications. Each of the applications is discussed below: 

> Crime prevention: Image recognition systems are used by police forces to find the criminals involved in a 
particular case. 

> Medical Diagnosis: Using CBIR in a medical database of medical images to aid diagnosis by identifying 
similar past cases. 

> Intellectual Property: Trademark image registration, where a new candidate mark is compared with existing 
marks to ensure no risk of confusing property ownership. It enforces compliance with the copyright 
regulations. 

> Military: This system can be used to identify whether a person belongs to Indian army or not. This is also 
used to find information about the destruction weapons used by the enemy countries. 

> Architectural and Engineering and Interior design: Reverse image search can be used to find what new 
technologies are used in those designs 

> Cultural heritage: We can find information about a snapshot of the famous traditions followed in foreign 
countries. 

IV. Results And Conclusion 

In this paper, an effective and efficient image retrieval approach is proposed. CBIR is an active 
research topic in image processing, pattern recognition and computer vision. This approach is based on the three 
well known algorithms: colour histogram, texture and moment invariants. The colour histogram is used to 
extract the colour features. Gabor filter is used to extract the texture features and moment invariant is used to 
extract the shape features of an image. It ensures that the proposed image retrieval approach produce results 
which are highly relevant to the content of query image, by taking into account the three distinct features of the 
image and similarity metrics based on Euclidean measure. Fuzzy similarity measures are used to find the similar 
images. 

V. Future Enhancements 

In this proposed system, only one image is captured and it is given as an input. Based on the input 
query, the desired results are produced. This project can be extended as specified below: 

> Multiple images of a person can be captured either through laptop camera or digital camera or phone 
camera. 

> The quality of those images can be affected by several reasons like environmental factors such as rainfall, 
fog or snow and movement of objects and usage of low resolution cameras. These factors obscure the image 
details and create noise in the image. In this case, Image preprocessing can be done to all the images to 
enhance their resolution. 

> After image preprocessing, face accuracy is calculated. Face accuracy determines which image has clear 
and accurate face in a maximum level. 

> Among the multiple images, we can select an image with maximum face accuracy and it can be given as an 
input. 

> Then the input query is compared with all the images in the database and similarity is calculated and similar 
images are produced as output with their details. 
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